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Abstract 

A stochastic model for behavioral changes by imitative pair interactions of indi- 
viduals is developed. 'Microscopic' assumptions on the specific form of the imitative 
processes lead to a stochastic version of the game dynamical equations. That means, 
the approximate mean value equations of these equations are the game dynamical 
equations of evolutionary game theory. 

The stochastic version of the game dynamical equations allows the derivation 
of covariance equations. These should always be solved along with the ordinary 
game dynamical equations. On the one hand, the average behavior is affected by 
the covariances so that the game dynamical equations must be corrected for in- 
creasing covariances. Otherwise they may become invalid in the course of time. On 
the other hand, the covariances are a measure for the reliability of game dynamical 
descriptions. An increase of the covariances beyond a critical value indicates a phase 
transition, i.e. a sudden change in the properties of the considered social system. 

The applicability and use of the introduced equations are illustrated by compu- 
tational results for the social self-organization of behavioral conventions. 

Keywords: evolutionary game theory, behavioral model, imitative processes, self-organ- 
ization of behavioral conventions, stochastic game theory, mean value equations, covari- 
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1 Introduction 



This paper treats a mathematical model for the temporal change of the proportions of in- 
dividuals showing certain behavioral strategies. Models of this kind are of special interest 
for a quantitative understanding or prognosis of social developments. For the description 
of the competition or cooperation in populations there already exist game theoretical ap- 
proaches (cf. e.g. VON Neumann and Morgenstern, 1944; Luce and Raiffa, 1957; 
Rapoport and Chammah, 1965; Axelrod, 1984). In order to cope with time- dependent 
problems the method of iterated games has been developed and used for a long time. How- 
ever, some years ago, the game dynamical equations have been discovered (TAYLOR and 
JONKER, 1978; HOFBAUER et. al, 1979; Zeeman, 1980). These are ordinary differen- 
tial equations, which are related to the theory of evolution (ElGEN, 1971; FiSHER, 1930; 
Eigen and Schuster, 1979; Hofbauer and Sigmund, 1988; Feistel and Ebeling, 
1989). Therefore, one also speaks of evolutionary game theory. 

The game dynamical equations have the following advantages: 

• They are continuous in time which is more adequate for many problems. 

• Ordinary differential equations are easier to handle than iterated formulations. 

• Analytical results can be derived more easily (cf. e.g. Hofbauer and Sigmund, 
1988; Helbing, 1992). 

Up to now, there only exists a 'macroscopic' foundation of the game dynamical equa- 
tions, i.e. a derivation from a collective level of behavior (cf. Section p.lj ). In this paper 
a 'microscopic' fonndaXion will be given, i.e. a derivation on the basis of the individual 
behavior. With this aim in view, we will first develop a stochastic behavioral model for 
the following reasons: 

• A stochastic model, i.e. a model that can describe random fluctuations of the quan- 
tities of interest, can cope with the fact that behavioral changes are not exactly 
predictable (which is a consequence of the 'freedom of decision-making'). 

• The phenomena appearing in the considered social system can be connected to the 
principles of individual behavior. As a consequence, processes on the 'macroscopic' 
(collective) level can be understood as effects of 'microscopic' (individual) interac- 
tions. 

• The probability of occurence of each strategy can be calculated. This is especially 
important for small social systems which are subject to large fluctuations (since 
they consist of a few individuals only). 

• The stochastic model allows the derivation of covariance equations (cf. Section 
Since the covariances influence the average temporal behavior, they are an essential 
criterium for the validity and reliability of behavioral descriptions by rate equations 
(which are actually approximate mean value equations). If the covariances exceed a 
certain critical value, this indicates the occurence of a phase transition, i.e. a sudden 
change of the properties of the considered social system. 
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For the description of systems that are subject to random fluctuations different stochastic 
methods have been developed (cf. e.g. Gardiner, 1983; Weidlich and Haag, 1983; 
Helbing, 1992). One method is to delineate the temporal evolution of the probabil- 
ity distribution over the different possible states (which represent behavioral strategies, 
here). This method is particularly suitable for an 'ensemble' of similar systems or for 
frequently occuring processes. In the case of discrete states, the master equation has to 
be used, whereas in the case of continuous state variables the Fokker-Planck equation 
is normally preferred since it is easier to handle (Fokker, 1914; Planck, 1917). The 
Fokker-Planck equation can, in good approximation, also be applied to systems with a 
large number of discrete states if state changes only occur between neighbouring states. It 
can be derived from the master equation by a Kramers-Moyal expansion (Kramers, 
1940; MoYAL, 1949), i.e. a second order Taylor approximation, then. 

Another method, the Langevin equation (1908) (or stochastic differential equation) is 
applied to the description of the temporal evolution of single fluctuation-affected sys- 
tems. It consists of a deterministic dynamical part which delineates systematic state 
changes and a stochastic fluctuation term which reflects random state variations. The 
Langevin equation can be reformulated in terms of a Fokker-Planck equation and 
vice versa (if the fluctuations are GAUSSian and 5-correlated which is normally the case; 
cf. Stratonovich, 1963, 1967; Weidlich and Haag, 1983). 

Although these methods come from statistical physics, the application to interdisciplinary 
topics has meanwhile a long and successful tradition, beginning with the work of Wei- 
dlich (1971, 1972), Haken (1975), Prigogine (1976), NicoLis and Prigogine (1977). 
Also for social and economic processes Fokker-Planck equation models (cf. e.g. Wei- 
dlich and Haag, 1983; Topol, 1991) as well as master equation models (cf. e.g. Wei- 
dlich and Haag, 1983; Weidlich, 1991; Haag et.al., 1993; Weidlich and Braun, 
1992) were proposed. In this paper we will develop a behavioral model on the basis of the 
master equation (Section |]). For this purpose we have to specify the transition rates, i.e. 
the probabilities per time unit with which changes of behavioral strategies take place. The 
transition rates can be decomposed into 



• rates describing spontaneous strategy changes, and 

• rates describing strategy changes due to pair interactions of individuals. 



In the following we will restrict our considerations to imitative pair interactions which seem 
to be the most important ones (Helbing, 1994). By distinguishing several subpopulations 
a, different types of behavior or different groups of individuals can be taken into account. 

In order to connect the stochastic behavioral model to the game dynamical equations the 
transition rates have to be chosen in such a way that they depend on the expected successes 
of the behavioral strategies (cf. Section |3]^) . The ordinary game dynamical equations are 



the approximate mean value equations of the stochastic behavioral model (cf. Section ^.2|) . 



For the approximate mean value equations correction terms can be calculated. These 
depend on the covariances (of the numbers of individuals pursuing a certain strategy) 



(cf. Section |4.1.4|) . Neglecting these corrections, the game dynamical equations may lose 



their validity after some time. The calculation of the covariances allows the determination 



3 



of the time interval during which game dynamical descriptions are reliable (cf. Section 



4.1.5). 



The introduced equations are illustrated by computational results for the self-organization 
of a behavioral convention by a competition between two alternative, but equivalent strate- 
gies (cf. Sections ^]3| and ^). These results are relevant for economics with respect to the 



rivalry between similar products (Arthur, 1988, 1989; Hauk, 1994). 



2 The stochastic behavioral model 

Suppose we consider a social system with individuals. These individuals can be divided 
into A subpopulations a consisting of A^^ individuals, i.e. 

a=l 

By subpopulations different social groups (e.g. blue and white collars) or different charac- 
teristic types of behavior are distinguished. In the following we will assume that individuals 
of the same subpopulation (group) behave cooperatively due to common interests, whereas 
individuals of different subpopulations (groups) do not so due to conflicting interests. 

The Na individuals of each subpopulation a are distributed over several states 

te{l,...,S} 

which represent the alternative (behavioral) strategies of an individual. For the time being, 
every individual shall be able to choose each of the S strategies, i.e. the same strategy 
set shall be available for each subpopulation. If the occupation number n^{t) denotes the 
number of individuals of subpopulation a who use strategy i at the time t, we have the 
relation 

E<W = iv„. (1) 

i=l 

Let 

n := {nl,...,ni,...,nj) 

be the vector consisting of all occupation numbers n". This vector is called the sociocon- 
figuration since it contains all information about the distribution of the individuals 
over the states i. P{n, t) shall denote the probability to find the socioconfiguration n at 
the time t. This implies 

0<P(n,t)<l and 5I^(^'^) = 1- 

n 

If transitions from socioconfiguration n to n' occur with a probability of P(n', /!: + At|n, t) 
during a short time interval At, we have a (relative) transition rate of 

P«t + At|n,t) 
win n; t) := lim . 

^ ' ' ^ Ai^O At 
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The absolute transition rate of changes from n to n' is the product w{n'\n]t)P{n,t) of 
the probabihty P{n, t) to have configuration n and the relative transition rate w{n'\n; t) 
if having configuration n. Whereas the inflow into n is given as the sum over all absolute 
transition rates of changes from an arbitrary configuration n' to n, the outflow from 
n is given as the sum over all absolute transition rates of changes from n to another 
configuration n'. Since the temporal change of the probability P{n, t) is determined by 
the inflow into n reduced by the outflow from n, we find the socalled master equation 

—P(n,t) = inflow into n — outflow from n 
dt ^ ^ 

= J2'^i^\'^''^t)P{n',t) -J2'^i^'\'^'^t)P{n,t) (2) 

n' n' 

(Pauli, 1928; Haken, 1979; Weidlich and Haag, 1983; Weidlich, 1991). 

It will be assumed that two processes contribute to a change of the socioconflguration n: 

• Individuals may change their strategy i spontaneously and independently of each 
other to another strategy i' with an individual transition rate ida{i'\i',t). These 
changes correspond to transitions of the socioconflguration from n to 

<,-=(n},...,«, + l),...,«-l),...,n|) 

with a configurational transition rate w{n^,^\n;t) = n°:Wa{i'\i'-,t) which is propor- 
tional to the number n°: of individuals who can change strategy i. 

• An individual of subpopulation a may change the strategy from i to i' during a pair 
interaction with an individual of some subpopulation h who changes the strategy 
from j to j'. Let transitions of this kind occur with a probability Wabii' , j] t) per 
time unit. The corresponding change of the socioconflguration from n to 

nJi':= (n},...,«, + l),...,«-l),...,(4 + l),...,(n;-l),...,n|) 



I ft] ■ V i' • ■ • ; v'j' I ■ ■ ■ 1 v-j ■ ■ ■ 1 \'"]' I ■ ■ ■ 1 



leads to a conflgurational transition rate w{n'}!'j,^j\n]t) = n°;n'jWab{i' , j]t) which 
is proportional to the number nfn^j of possible pair interactions between individu- 
als of subpopulations a and b who pursue strategy i and j respectively. (Exactly 
speaking — in order to exclude self-interactions — n'^n'^iSaaii' , j'lhi'it) has to be re- 
placed by n^{n'^ — l)waa{i' , j'lh'^'it) if '>^aa{i' , j'\h'i]t) <^ Wa{i'\i]t) is invalid and 
P{n,t) is not negligible where ^ 1 is not fulfllled.) 

The resulting conflgurational transition rate w{n'\n;t) is given by 

{nfida{i'\i;t) if n' = n'^,^ 

n^npab{^',f\^,J■,t) if n' = nf-,,- (3) 
otherwise. 

As a consequence, the explicit form of master equation (0) is 
d 



, P{n,t) = ^ [«, + lR(zK';t)PK,,,,t)-<z2,(^'K;t)P(n,ty 

+ I E E [«' + + l)wab{^,J\^',f■,t)P{nf^,,,,t) 

Kn';Wab{i',f\i,J]t)P{n,t)] (4) 
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(cf. Helbing, 1992a). 

We restricted our considerations to pair interactions, here, since they normally play the 
most significant role. Even in groups the most frequent interactions are alternating pair 
interactions — not always but in many cases. In situations where simultaneous interactions 
between more than two individuals are essential (one example for this is group pressure), 
the above master equation must be extended by higher order interaction terms. The 
corresponding procedure is discussed by Helbing (1992, 1992a). 



3 Stochastic version of the game dynamical equa- 
tions 

3.1 Specification of the transition rates 

The pair interactions 

^',f^h3 (5) 

of two individuals of subpopulations a and b who change their strategy from i and j to 
i' and j' respectively can be classified into three different kinds of processes: Imitative 
processes, avoidance processes, and compromising processes. These are discussed in detail 
and simulated in several publications (Helbing, 1992, 1992b, 1994). In the following we 
will focus to imitative processes (processes of persuasion) which describe the tendency to 
take over the strategy of another individual. These are of the special form 

hi' — hj (^T^j), (6a) 
jj' — iJ {i^j)- (6b) 

The corresponding pair interaction rates read 

Wab{i',j'\i,j;t) = VabPba{i\j;t)Sii'6ij'{l - 6ij) (7a) 

+ l^abPabUli; (1 - 5ij) , (7b) 

where the Kronecker symbol Sij is defined by 



5ij :-- 



1 if i = j 
ifz^j. 



The factors (1 — 6ij) result from the constraint i ^ j, whereas factors of the form 6ij 
correspond to conditions of the kind i = j which follow by comparison of (^) and (^d) 
respectively with (|S|). The parameter 

l'ab-=NbVab (8) 

represents the contact rate between an individual of subpopulation a with individuals of 
subpopulation b. Pab{.3\h t) denotes the probability of an individual of subpopulation a to 
change the strategy from i to j during an imitative pair interaction with an individual of 
subpopulation b, i.e. 
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For j i we will assume 

Pab{j\i; t) := fabRa{j\i; t) (9) 

where the parameter is a measure for the frequency of imitative pair interactions be- 
tween individuals of subpopulation a when confronted with an individual of subpopulation 
b. Ra{j\i; t) is a measure for the readiness of individuals belonging to subpopulation a to 
change the strategy from i to j during a pair interaction. 



3.2 'Microscopic' foundation of evolutionary game theory 

The problem of this section is to specify the frequency fah and the readiness Ra{i\v,t) = 
Ra{j\i] n; t) in an adequate way. For this we make the following assumptions: 

• By experience each individual knows — at least approximately — the expected success 
of the strategy used: We will define the expected success of a strategy i for an 
individual of subpopulation a in interactions with other individuals by 




Here, the parameter 

c 

represents the relative contact rate of an individual of subpopulation a with indi- 
viduals of subpopulation b. n'j{t)/Ni) is the probability that an interaction partner 
of subpopulation b uses strategy j. Eab{i,j) is an exogenously given quantity that 
denotes the success of strategy i for an individual of subpopulation a during an in- 
teraction with an individual of subpopulation b who uses strategy j. Since all these 
quantities can be determined by each individual, the evaluation of the expected 
success Ea{i,t) is obviously possible. 

• In interactions with individuals of the same subpopulation an individual tends to 
take over the strategy of another individual if the expected success would increase: 
When an individual who tiscs strategy i meets another individual of the same sub- 
population who uses strategy j, they compare their expected successes Ea{ht) and 
Ea{j,t) respectively by exchange of their experiences. (Remember that individuals 
of the same subpopulation were assumed to cooperate.) The individual with strat- 
egy i will imitate the other's strategy j with a probability Pab{j\i] t) that is growing 
with the expected increase 

AjiEa -.^ Ea(j,t) - Ea(i,t) 

of success. If a change of strategy would imply a decrease of success {AjiEa < 0), the 
individual will not change the strategy i. Therefore, the readiness for replacing the 
strategy i by j during an interaction within the same subpopulation can be assumed 
to be 

RaUli; t) := max t) - E^ii, t), o) (11) 

where max(a;, y) is the maximum of the two numbers x and y. This describes an 
individual optimization or learning process. 
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• In interactions with individuals of other subpopulations (who behave in a non- 
cooperative way) normally no imitative processes will take place: During these in- 
teractions the expected success Eb{j,t) of the interaction partner can at best be 
estimated by observation since he will not tell his experiences. Moreover, due to 
different criteria for the grade of success, the expected success of a strategy j will 
normally be varying with the subpopulation (i.e. Eaii^t) ^ Ej,{i,t) for a ^ b). As a 
consequence, an imitation of the strategy of individuals belonging to another sub- 
population would be very risky since it would probably be connected with a decrease 
of expected success. Hence the assumption 

/afe:=5a6 = | I (12) 

will normally be justified. 

Relation (|12D also results in cases where the strategies of the respective other sub- 
populations cannot be imitated due to different (disjunct) strategy sets. Then, we 
need not to assume that individuals of the same subpopulations cooperate, whereas 
individuals of different subpopulations do not. 

In Section ^ it will turn out that the game dynamical equations are the approximate mean 
value equations of the stochastic behavioral model defined by (0) to ([T^p. In this sense, 
the model of this section can be regarded as stochastic version of the game dynamical 
equations. Moreover, the assumptions made above are a 'microscopic' foundation of evo- 
lutionary game theory since they allow a derivation of the game dynamical equations on 
the basis of individual behavior patterns. 



3.3 Self-organization of behavioral conventions by competition 
between strategies 

As an example for the stochastic game dynamical equations we will consider a case with 
one subpopulation only {A = 1). In this case we can omit the indices a, b, and the sum- 
mation over b. Let us assume the individuals to choose between two equivalent strategies 
i G {1,2}, i.e. the success matrix E_ = (^E{i,j)j is symmetrical: 

According to the relation 

ni{t) + n2{t) = N 

(cf. ([1|)), n2(t) = N — ni(t) is already determined by ni(t). For spontaneous strategy 
changes due to trial and error we will take the simplest form of transition rates: 

wU\z;t):=W. (14) 

A situation of the above kind is the avoidance behavior of pedestrians (cf. Helbing, 
1991, 1992): In pedestrian crowds with two opposite directions of motion, pedestrians 
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have sometimes to avoid each other in order to exclude a colhsion. For an avoidance 

maneuver to be successful, both pedestrians concerned have to pass the respective other 

pedestrian either on the right hand side (strategy i = 1) or on the left hand side (strategy 

i = 2). Otherwise, both pedestrians have to stop (cf. Figure ||a). Here, both strategies are Fig. 1 

equivalent, but the success of a strategy increases with the number rii of individuals who 

use the same strategy. In success matrix (|^) we have 

C > 0, 

then. 

Empirically one finds that the probability Pi of choosing the right hand side is usually 
different from the probability P2 = 1 ~ -Pi of choosing the left hand side. Consequently, 
opposite directions of motion normally use separate lanes (cf. Figure |l|b). 

We will now examine if our behavioral model can explain this symmetry breaking (the 
fact that Pi 7^ P2). Figure ^ shows some computational results for C = 1 and different 
values of W/u. If Fig. 2 

AW 

«::=!- — <0, (15) 

the configurational distribution is unimodal and symmetrical with respect to ni = N/2 = 
n2, i.e. both strategies will be chosen by about one half of the individuals. At the critical 
point K = there appears a phase transition (bifurcation). This is indicated by the broad- 
ness of the probability distribution P{n, t) = P{ni,n2] t) = P{ni, N — ni]t) which comes 
from socalled critical fluctuations (cf. Haken, 1983). The term 'critical fluctuations' de- 
notes the fact that the fluctuations become particularly large at a critical point since the 
system behavior is unstable, then. Whereas the individuals behave more or less indepen- 
dently before the phase transition (/t < 0), around the critical point the individuals begin 
to act correlated due to their (imitative) interactions. However, the spontaneous strategy 
changes (represented by W) still prevent the formation of a behavioral preference. Above 
the critical point (i.e. for n > 0) the correlation of individual behaviors is strong enough 
for the self- organization (emergence) of a behavioral convention: The configurational dis- 
tribution becomes multimodal in the course of time with maxima at ni 7^ N/2 so that one 
of the two equivalent strategies will very probably be chosen by a majority of individuals. 
In this connection one also speaks of symmetry breaking (Haken, 1979, 1983). 

Behavioral conventions often obtain a law-like character after some time. Which one 
of two equivalent strategies will win the majority is completely random. It is possible, 
that conventions differ from one region to another. This is, for example, the case for the 
prescribed driving direction of cars. 

The model of this section can also be applied to the competition between the two video 
systems VHS and BETA MAX which were equivalent with respect to technology and 
price at the beginning (Hauk, 1994). In the course of time VHS won this rivalry since 
(for reasons of compatibility concerning copying, selling or hiring of video tapes) it was 
advantageous for new purchasers to decide for that video system which gained a small 
majority at some moment. Other examples for the emergence of a behavioral convention 
are the revolution direction of clock hands, the direction of writing, etc. A generalization 
of the above model to the case of more than two alternative strategies is easily possible. 
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Of course, the model can also be adapted to situations where one behavioral alternative is 
superior to the others. However, the formation of a behavioral convention is trivial, then. 

Finally, some related models should be mentioned which were proposed during the re- 
cent years for the description of symmetry breaking phenomena in economics: Orlean 
(1992, 1993) and Orlean and Robin (1992) presented a phase transition model using 
polynomial transition rates which base on a BAYESian rationale. Durlauf (1989, 1991) 
used MARKOVian fields to explain the non-ergodic (i.e. path-dependent) behavior of some 
economic systems. Follmer (1974) applied the IsiNG model paradigm (1925) to model 
an economy of many interacting agents and discussed under which conditions a symme- 
try breakdown occurs. A similar model for polarization effects in opinion formation was 
already suggested by Weidlich (1972). Last but not least ToPOL (1991) presented a 
Fokker-Planck equation model for the explanation of bubbles in stock markets by 
mimetic contagion (i.e. some kind of imitative interactions) between agents. 



4 Most probable and expected strategy distribution 

Because of the huge number of possible socioconfigurations n, in more complex cases than 
in Section |3]^ the master equation for the determination of the configurational distribution 
P{n,t) is usually difficult to solve (even with a computer). However, 

• in cases of the description of single or rare social processes the most probable strategy 
distribution 

Pm := ^ (16) 

is the quantity of interest whereas 

• in cases of frequently occuring social processes the interesting quantity is the expected 
strategy distribution 

Ptit):=^^. (17) 

P^{t) is the proportion of individuals within subpopulation a using strategy i so that 

mt)>0 and J2Pnt) = l. 

i 

Equations for the most probable occupation numbers n'^{t) can be deduced from a Langevin 
equation (1908) for the temporal development of the socioconfiguration n{t). For the mean 
values {n1)t of the occupation numbers normally only approximate closed equations can 
be derived. A measure for the reliability of n^{t) and (n")t with respect to the possible 
temporal developments of n"(t) are the variances o"j"'^(t) of nf^it). If the standard devia- 



tion \J(Tf°'{t) becomes comparable to 0.12n"(t) or 0.12(n")i, the values of n°;{t) and {n°;)t 
respectively are not representable for n"(t) any more (cf. Section [4.1. 5|) . In the case of 



P(n, t) being normally distributed this would imply a probability of 34% (5%) that the 
value of n^{t) deviated more than 12% (24%) from n^(t) and (n")t respectively. Moreover, 
if the variances (Tii{t) become large, this may indicate a phase transition, i.e. a non-ergodic 
(path-dependent) temporal evolution of the system (see Figure |^). 
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4.1 Mean value and covariance equations 

The mean value of a function f{n,t) is defined by 

(/(n, t))t ^ {f{n, t)) := t)P{n, t) . 

n 

From master equation (H) can be derived that the mean values of the occupation numbers 
f{n,t) = n1 are determined by the equations 

^ = K(n,t)) (18) 

with the drift coefficients 

m^{n,t) := - n^)w{n'\n;t) 



J2 - uf{i'\i; tX] (19) 



and the effective transition rates 



w 



t) := + E E E ^-b{t', f\^, 3\ t)n] (20) 

b j' j 



(cf. Helbing, 1992, 1992a). Obviously, the contributions Wab{i',j'\i,j',t)nj due to pair 
interactions are proportional to the number of possible interaction partners. 

4.1.1 Approximate mean value equations 

Equations ( plSf ) are no closed equations, since they depend on the mean values (n"n^), 
which are not determined by ([181) . We have, therefore, to find a suitable approximation. 
Using a first order Taylor approximation we obtain the approximate mean value equa- 
tions 

= <({n>,i). (21) 

These are applicable if the configurational distribution P{n,t) has only small covariances 

< := («-«))K'-^))) 

= «n5)-«)(n;)^0. (22) 

Condition ( ^2]) corresponds to the limit of statistical independence {nfn'j) = (nf)(n^) of 
the occupation numbers (and, therefore, of the individual behaviors). 
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4.1.2 BoLTZMANN-like equations 

Inserting (0), (|T9|) , and (|^) into ( ^iD the resulting approximate equations for the ex- 
pected strategy distribution P^{t) are 

jP-it) = E k(^K'; t)Pm - t)Pm] (23) 

i' 

with the mean transition rates 



w 



t) = w,{z% t) + E E E f\^, j; t)P;{t) . (24) 

j' j 



Equations (||), (||) are called BOLTZMANN-/zA;e equations (BOLTZMANN, 1964; Helbing, 
1992, 1992a) since the mean transition rates (^) depend on the strategy distributions 
P^{t) due to pair interactions. Assuming (|^), (||), and (P) we obtain the formula 

w^'iili'; t) = iua{t\i'; t) + Raii\i'; t) E i^abfabP-it) (25) 

6 

with Ra{i\i'; t) := Ra{i\i'', (n); t) for the mean transition rates. (p3|) and (p5|) are a special 
case of more general equations introduced by Helbing (1992, 1992b, 1994) for the tem- 
poral development of the expected strategy distribution in a social system consisting of a 
huge number ^ 1 of individuals. 



4.1.3 Approximate covariance equations 

In many cases, the configuration no at an initial time to is known by empirical evaluation, 
i.e. the initial distribution is 

P(n,to) = Snno ■ 

As a consequence, the covariances afj vanish at time to and remain small during a certain 
time interval. For the temporal development of afj, the equations 

= {mf^M) + (« - «))m;(n,t)) + {{n] - {n';))mnn,t)) (26) 
can be derived from master equation (^ (cf. Helbing, 1992, 1992a). Here, 



rn 



ab I 



n,t) := E«-<)K'-^5V(^V;^) 



j 

- [n'^,w''{i\i';t) +n^w''{i'\t;t) 

E E [^>S'W„b(i, i'\3,f; t) + n1nlwab{j,j'\i, i'; t) 
j' j 

EE [n>',>Wabij,t'\t,3';t) + n]n'l,Wabi^,3V,^';t)] (27) 
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are diffusion coefficients. Equations ( pq) are again no closed equations. However, a first 
order TAYLOR approximation of the drift and diffusion coefficients m '(n, t) leads to the 
equations 

(cf. Helbing, 1992, 1992a) which are solvable together with (PT|). The approximate co- 
variance equations (pS]) allow the determination of the time interval during which the 
approximate mean value equations (|2l|) are valid (cf. Section [4.1.5| and Figures [Sb]) . 
They are also useful for the calculation of the reliability (or representativity) of descrip- 
tions made by equations (pi]) . Moreover, they are necessary for corrections of approximate 
mean value equations (|21|). 



4.1.4 Corrected mean value and covariance equations 

Equations (^T]) and (^8]) are only valid for the case 

(29) 

where the absolute values of the covariances a^^ are small, i.e. where the configurational 
distribution P{n,t) is sharply peaked. For increasing covariances a better approximation 
of (|T^) and ( PUD should be taken. A second order Taylor approximation of (|T^) and (PB| ) 
respectively results in the corrected mean value equations 

dt ^"^^^^"^'') + 2^^"^-a(n5)9(.|) 
and the corrected covariance equations 



^ ~ n^^''((n)t) + WT.^'^^^^SM2^ 
dt - "^-^^""^^'^^^ItV'' dK)d{nf) 




dm){{n),t) dmt{{n),t) \ 

+ ^'"'-^ dK) +"^- dK) ) 

(Helbing, 1992, 1992a). Note that the corrected mean value equations explicitly depend 
on the covariances a^^, i.e. on the fluctuations due to the stochasticity of the processes 
described. They cannot be solved without solving the covariance equations. A comparison 
of (|30|) with (21) shows that the approximate mean value equations only agree with the 



corrected ones in the limit afj of negligible corvariances (cf. also (^)). However, the 
calculation of the covariances is always recommendable since they are a measure for the 
reliability (or representativity) of the mean value equations. If the covariances become 
large in the sense of equation (|33|) this may indicate a phase transition. 
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4.1.5 Computational results 



A comparison of exact, approximate and corrected mean value and variance equations is 
given in Figures ^ to |^. These show computational results corresponding to the example 
of Section p.3| (cf. Figure 0). Exact mean values (rii) and variances an are represented by 
solid lines whereas approximate results according to (^), (|28| ) are represented by dotted 
lines and corrected results according to (pOf), (pTJ) by broken lines. Fig. 3-4 



For K > the approximate mean value equations (^) become useless since the variances 
are growing due to the phase transition. As expected, the corrected mean value equations 
yield better results than the approximate mean value equations and they are valid for a 
longer time interval. 

A criterium for the validity of the approximate equations (|21|), ( ]28|) and the corrected 
equations (pO]), (pTF) respectively are the relative central moments 



/ (n°-^ _ in"'-)) ■ ■ in''"' - (n"'"))) 

Whereas the approximate equations (0), (^) already fail, if 

\Cmit)\<OM (32) 

is violated for m = 2 (compare to (|29|), (|22|)), the corrected equations (pO|), ( |3TD presuppose 
condition (^) only for 3 < m < / with a certain, well-defined value / (cf. Helbing, 1992, 
1992a for details). However, even the corrected equations (0), (|3TD become useless if the 
probability distribution P{n, t) becomes multimodal, i.e. if a phase transition occurs. This 
is the case if 



\C2it)\ 



< 0.12 (33) 



is violated (cf. Figure ^. Fig. 5a,b 



4.2 Equations for the most probable strategy distribution 

After the transformation of master equation (|^) into a Fokker-Planck equation by a 
second order Taylor approximation, it can be reformulated in terms of a Langevin 
equation (1908) (cf. Weidlich and Haag, 1983; Helbing, 1992). The latter reads 

^n'^it) ^=^^ m^(n, t) + fluctuations (34) 

and describes the temporal development of the socioconfiguration n{t) in dependence of 
process immanent fluctuations (that are determined by the diffusion coefficients m^j). As 
a consequence, 

jnm''='mtin,t) (35) 

are the equations governing the temporal development of the most probable occupation 
numbers n'^{t). Equations ( ^5|) look exactly like approximate mean value equations (^T]) . 



Therefore, if iY^ ^ 1, the approximate mean value equations have an interpretation even 
for large variances since they also describe the most probable strategy distribution. 
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5 The game dynamical equations 



5.1 'Macroscopic' derivation 



Before we will connect the stochastic behavioral model to the game dynamical equa- 
tions, we will discuss their derivation from a collective level of behavior. Let Ea{i,t) : = 
Ea{i, (n); t) be the expected success of strategy i for an individual of subpopulation a and 



(36) 



the mean expected success. If the relative increase 

dP^/dt 

of the proportion Pf(t) is assumed to be proportional to the difference [Ea{i,t) — Ea(t)] 
between the expected and the mean expected success, one obtains the game dynamical 
equations 

d 



-P^{t) = UaP^{t)[ES.t)-E,{t) 



(37) 



According to these equations the proportions of strategies with an expected success that 
exceeds the average Ea{t) are growing, whereas the proportions of the remaining strategies 
are falling. For the expected success Ea{i,t) one often takes the form 



b j 



(38) 



where the quantities Aab{i,j) have the meaning of payoffs which are exogeneously deter- 
mined. Consequently, the matrices 

A^h := [Aab{i,j)) 

are called payoff matrices. Inserting (p6D and (|38| ) into (|37D, one obtains the explicit form 



dt 



pnt) = VaPm 



Y.Aab{^,3)P'^{t)-Y.T.P?(i)^ab{^\3)P'^{t) 

b,j i' b,j 



(39) 



of the game dynamical equations. Equations of this kind are very useful for the investi- 
gation and understanding of the competition or cooperation of individuals (cf. e.g. HoF- 
BAUER and SiGMUND, 1988; Schuster et.al., 1981). Due to their nonlinearity they may 
have a complex dynamical solution, e.g. an oscillatory one (Hofbauer et.al., 1980; HoF- 
BAUER and SiGMUND, 1988) or even a chaotic one (Schnabl et.al., 1991). 



A slightly generalized form of 



d 



+ VaP-{t) Ea{l,t) - Ea{t) 



(40a) 
(40b) 
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is also known as selection mutation equation (Hofbauer and SiGMUND, 1988): ([40b| ) can 
be understood as effect of a selection (if Ea{i,t) is interpreted as fitness of strategy i), 
and ( [40 a| ) can be understood as effect of mutations. Equation (^) is a powerful tool 
in evolutionary biology (cf. EiGEN, 1971; Fisher, 1930; EiGEN and Schuster, 1979; 
Hofbauer and Sigmund, 1988; Feistel and Ebeling, 1989). In game theory, the 
mutation term could be used for the description of trial and error behavior or accidental 
variations of the strategy. 



5.2 Derivation from the stochastic behavioral model 

In this section we will look for a connection between the stochastic behavioral model of 
Section ^ and the game dynamical equations. For this purpose we compare the approxi- 
mate mean value equations of this stochastic behavioral model, i.e. the BOLTZMANN-like 
equations (0), ( pSj) with the game dynamical equations (|iO|). Both equations will be 
identical only if 

l^abfab = ^a^ab ■ 



This condition corresponds to (|T2D if 



= V, 



a ^ aa 



Inserting assumptions (|T0|) to (|^) into the BoLTZMANN-like equations (pSf) , (^) the 
game dynamical equations ( ^0]) result. We have only to introduce the identifications 

Kbii.j) ■= rabEab{i,j) , 
E^{t,t) := Fail, (n);t) = 5]^r,feE,fe(z, j)P,'(t) , 

b j 

and to apply the relation 

max (e,(z, t) - EaU, t), o) - max (e,(j, t) - E^ii, t), o) = E^ii, t) - E^U, t) . 

The game dynamical equations (including their properties and generalizations) are more 
explicitly discussed elsewhere (Helbing, 1992). An interesting application to a case with 
two subpopulations can be found in the book of Hofbauer and Sigmund (1988: pp. 
137-146). In the following, we will again examine the example of Section |3.3| where we 
have one subpopulation and two equivalent strategies. The game dynamical equations 
(^) corresponding to (|13]) and (|l^ have, then, the explicit form 

m) = -2 (p,{t) -l]\w+ ucp^it) (m) - i)i . (41) 



dt ' V ' ' 2 

According to (0), Pi = 1/2 is a stationary solution. This solution is stable for 

AW 

i.e. if spontaneous strategy changes are dominating and, therefore, prevent a self-organ- 
ization process. 
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At the critical point k = symmetry breaking appears: For k, > the stationary solution 
Pj = 1/2 is unstable and the game dynamical equations (|4l|) can be rewritten in the form 



IpM ^ -2 (m - i) (m - (m - i^) . (42) 

This means, for k > we have two additional stationary solutions Pj = (1 + ^/K,)/2 and 
Pj = (1 — a/k)/2 which are stable. Depending on initial fluctuations, one strategy will 
win a majority of 100 ■ ^/K percent. This majority is the greater the smaller the rate W 
of spontaneous strategy changes is. 



6 Modified game dynamical equations 

At first glance the crease of P{ni,N — ni;t) at rii = N/2 = n2 in the illustrations 
of Figure Q appears somewhat surprising. A mathematical analysis shows that this is a 
consequence of the crease of the function Ra{j\i; t) = ma.x{Ea{j, t) — Ea{i, t),0). It can be 
avoided by using the modified approach 

RaU, t) := ^ exp[Ea{j, t) - Kit, t)] . (43) 

(^) also leads to a phase transition for k = (cf. Figure ^ and very similar results for the 
approximate mean value equations since the game dynamical equations result as Taylor 
approximation of those. According to (^3]), imitative strategy changes from i to j will again Fig. 6 
occur the more frequent the greater the expected increase AjiEa = Ea{j,t) — Ea{i,t) of 
success is. 

Approach (^) originally stems from physics where the exponential function for the tran- 
sition probability is due to the need to obtain the Boltzmann distribution (1964) as 
stationary distribution. Its application to behavioral changes was suggested by Weidlich 
(1971, 1972) in connection with a IsiNG-like (1925) opinion formation model. Meanwhile, 
related models were also proposed for economic systems (Haag et. al., 1993; Weidlich 
and Braun, 1992; DuRLAUF, 1989, 1991). In contrast to this, Orlean (1992, 1993) 
and Orlean and Robin (1992) prefer a transition probability which has the form of a 
polynomial of degree two and bases on a BAYESian rationale. 



The advantage of (^31) is that it guarantees the non-negativity of Ra{j\i', t). Moreover, the 
exponential approach factorizes into a pull-term exp[Ea{j, t)] and a push-term exp[—Ea{h t)]- 
Relevant for strategy changes is not the absolute success Ea{j,t) of an available strategy 
j but the relative success [Ea{j,t) — Ea{ht)] with respect to the pursued strategy i. 



Furthermore, approach ( ^3]) can be related to a decision theoretical model for choice under 
risk. For this let us assume that the utility of a strategy change from i to j is given by a 
known part 

UaiMt) := [EaU^t] - Eait^t) 

and an unknown part ej (i.e. an error term) which comes from the uncertainty about the 
exact value of [Ea{j,t) — Eaiht)] (since Eaiht) like is subject to fluctuations). If the 
individual choice behavior is the result of a maximization process (i.e. if an individual 
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chooses the alternative j for which Ua{j\i;t) + ej > Ua{i'\i]t) + e^/ holds in comparison 
with all other available alternatives i') and if the error terms are identically and indepen- 
dently Weibull distributed, the choice probabilities Pa{j\i] t) are given by the well-known 
multinomial logit model (Domencich and McFadden, 1975). It reads 

. _ exp[Ea{j,t) - Ea{i,t)] 
i' 

(For a more detailed discussion cf. Helbing, 1992.) 

Approach (l43|) can also be derived by entropy maximization (Helbing, 1992) or from the 
law of relative effect in combination with the FECHNERzan law of psychophysics (Luce, 
1959; Helbing, 1992). 



7 Summary and Outlook 



A quite general model for changes of behavioral strategies has been developed which takes 
into account spontaneous changes and changes due to pair interactions. Three kinds of 
pair interactions can be distinguished: imitative, avoidance and compromising processes. 
The game dynamical equations result for a special case of imitative processes. They can 
be interpreted as equations for the most probable strategy distribution or as approximate 
mean value equations of a stochastic version of evolutionary game theory. In order to 
calculate correction terms for the game dynamical equations as well as to determine 
the reliability or the time period of validity of game dynamical descriptions, one has to 
evaluate the corresponding covariance equations. Therefore, covariance equations have 
been derived for a very general class of master equations. 

The model can be extended in a way that takes into account the expectations about the 
future temporal evolution of the expected successes Ea{i,t) (the 'shadow of the future). 
For this purpose, in ( ^OD Ea{i, t) must be replaced by a quantity E*{i, t) which represents 
the expectations about the future success of strategy i on the basis of its success Ea{i, t') 
at past times t' < t. Different ways of mathematically specifying the future expectations 
E*{i,t) were discussed by ToPOL (1991), Glance and Huberman (1992) as well as 
Helbing (1992). 
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4 

Figure 1: (a) For pedestrians with an opposite direction of motion it is advantageous if both prefer either 
the right hand side or the left hand side when trying to pass each other. Otherwise, they would have to 
stop in order to avoid a collision. The probability Pi of choosing the right hand side is usually different 
from the probability P2 = 1 — Pi of choosing the left hand side. 

(b) Opposite directions of motion normally use separate lanes. Avoidance maneuvers are indicated by 
arrows. 
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Figure 2: Probability distribution P(n, t) = P{ni, N — ni;t) of the socioconfiguration n for varying values 
of the control parameter k. For k = a phase transition occurs: Whilst for n < both strategies are used 
by about one half of the individuals, for k > very probably one of the strategies will be prefered after 
some time. That means, a behavioral convention develops by social self-organization. 
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Figure 3: Exact ( — ), approximate (• • •) and corrected ( ) mean values (upper curves) and variances 

(lower curves) for a small configurational distribution P(n, t): The simulation results for the approximate 
equations are acceptable, those for the corrected equations very well. 
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Figure 4: As Figure ^, but for a broad configurational distribution: The corrected equations still yield 
useful results, whereas the approximate equations already fail since the variances are not negligible. 
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Figure 5a: As Figure |3[ but for a multimodal configurational distribution: Not only the approximate but 
also the corrected equations fail after a certain time interval. However, whereas the approximate mean 
value and variance become unreliable already for i > 1, the corrected mean value and variance remain 
valid as long as t < 3. 
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Figure 5b: The relative central moments Cm{t) are a criterium for the validity of the approximate re- 
spectively the corrected mean value and covariance equations: If |C2(t)| ( — ) exceeds the value 0.04, the 
approximate equations fail. The corrected equations fail if |C3(t)| ( — ) or |C4(t)| (• • •) exceed the value 
0.04. This is the case if |C2(t)| becomes greater than 0.12 (indicating a phase transition). 
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Figure 6: Probability distribution P(n, t) = P{ni, N — ni\t) of the socioconfiguration n according to the 
modified stochastic game dynamical equations. The results are similar to those in Figure |^. For k < 
both strategies are used by about one half of the individuals, for k > very probably one of the strategies 
will be prefered after some time. Again, for k = a phase transition occurs. 
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